1. Field of the Invention
The present invention relates to a computer program product, system, and method for allocating physical nodes for processes in an execution plan.
2. Description of the Related Art
To run an application job involving multiple processes that execute in parallel, such as an Extract Load and Transform (“ETL”) job, a job executor reads a description of the configuration file template to determine the number of nodes on which processes will run. The job executor requests an allocation of nodes to run the job from a resource management tool, which then sends the job executor the node names that may be selected for the job. A user may then prepare a configuration file with actual node names selected from those provided by the resource management tool. The configuration file may allocate to different nodes a physical node, also known as a “fastname”, and a pool to which the node is assigned. The configuration file thus specifies actual physical resources to allocate in advance to the job before the execution plan is generated.
Using the configuration file, an execution plan may then be generated from the job description indicating the parts or partitions of the processes of the job to run on the selected nodes identified in the configuration file. The execution plan is then run to execute the processes on the selected node.
With ETL jobs, the processing may be partitioned at the nodes, where a conductor drives the entire job. Under the conductor there are section leaders, one section leader for each node, and there are multiple players for each section leader, or independent processes, that run on the same node as the section leader. Thus, the execution plan will require that players for the section leader are all assigned to the same node, which will comprise the physical node specified in the configuration file.